keepalived
==========================================
官网：http://www.keepalived.org/

推荐博客：
    http://outofmemory.cn/wiki/keepalived-configuration





    vrrp协议：
            Virtual Redundant Routing Protocol 
            
            虚拟路由器、VRID（虚拟路由器标识，0-255）、Master(Active)、Backup(Passive)、VIP、VMAC（00-00-5e-00-01--{VRID})
            工作方式：
                抢占式
                非抢占式
            工作模式：
                主备
                主/主：配置多个虚拟路由器（主备）；
            认证方式：
                无认证；
                简单字符串认证：预共享密钥；
                md5认证；
                    
keepalived:
    是vrrp协议的实现，原生设计目的为高可用ipvs服务；ka能够配置文件中的定义生成ipvs规则，并能够对各RS的健康状态进行检测；vrrp_script, vrrp_track；
        
    组件：
        控制组件：配置文件分析器
        内存管理
        IO复用
        核心组件：
            vrrp stack
            checker
            ipvs wrapper
            watch dog

    HA Cluster配置的前提：
        (1) 各节点时间同步；（CentOS 6：ntp协议，CentOS 7：chrony）
        (2) 确保iptables及selinux不会阻碍；
        (3) 各节点之间可通过主机名互相通信，名称解析服务的解析结果必须与“uname -n”命令的结果一致；（对ka并非必须）
        (4) 各节点之间的root用户可以基于密钥认证的ssh通信（对ka并非必须）；


安装keepalived：
        CentOS 6.4+，程序包已经在base源提供；

        主配置文件：/etc/keepalived/keepalived.conf
        Unit  file：/usr/lib/systemd/system/keepalived.service
        配置文件：/etc/sysconfig/keepalived


keepalived配置
    （1）：GLOBAL CONFIGURATION
    （2）：VRRPD CONFIGURATION
    （3）：LVS CONFIGURATION

（1）GLOBAL CONFIGURATION
        global_defs {
            # 通知到邮件
            notification_email {
               admin@example1.com
               ...
           }
            # 发送者邮箱
            notification_email_from admin@example.com
            # 邮件服务器
            smtp_server 127.0.0.1        # IP
            # 邮件链接超时（integer, seconds）
            smtp_connect_timeout 30
            # 
            router_id my_hostname        # string identifying the machine,
                                         # (doesn't have to be hostname).
            # 
            vrrp_mcast_group4 224.0.0.18 # optional, default 224.0.0.18
            # 
            vrrp_mcast_group6 ff02::12   # optional, default ff02::12
            # 开启 snmp 服务器分离
            enable_traps                 # enable SNMP traps
        }
Static routes/addresses
       keepalived can configure static addresses and routes. These addresses are NOT moved by vrrpd, they stay  on  the  machine.
       If you already have IPs and routes on your machines and your machines can ping each other, you don't need this section.

       The syntax is the same as for virtual addresses and virtual routes.

        static_ipaddress
        {
        192.168.1.1/24 dev eth0 scope global
        ...
        }

        static_routes
        {
        192.168.2.0/24 via 192.168.1.100 dev eth0
        ...
        }


（2）：VRRPD CONFIGURATION：包含：VRRP synchronization group(s) 和 VRRP instance(s)

VRRP synchronization group(s)
        #string, name of group of IPs that failover together
        vrrp_sync_group VG_1 {
           group {
             inside_network   # name of vrrp_instance (below)
             outside_network  # One for each moveable IP.
             ...
           }

           # notify scripts and alerts are optional
           #
           # filenames of scripts to run on transitions
           # can be unquoted (if just filename)
           # or quoted (if has parameters)
           # to MASTER transition
           notify_master /path/to_master.sh
           # to BACKUP transition
           notify_backup /path/to_backup.sh
           # FAULT transition
           notify_fault "/path/fault.sh VG_1"

           # for ANY state transition.
           # "notify" script is called AFTER the
           # notify_* script(s) and is executed
           # with 3 arguments provided by keepalived
           # (ie don't include parameters in the notify line).
           # arguments
           # $1 = "GROUP"|"INSTANCE"
           # $2 = name of group or instance
           # $3 = target state of transition
           #     ("MASTER"|"BACKUP"|"FAULT")
           notify /path/notify.sh

           # Send email notifcation during state transition,
           # using addresses in global_defs above.
           smtp_alert
        }




VRRP instance(s)
       describes  the moveable IP for each instance of a group in vrrp_sync_group.  Here are described two IPs (on inside_network
       and on outside_network), on machine "my_hostname", which belong to the group VG_1 and which will  transition  together  on
       any state change.

        #You will need to write another block for outside_network.
        vrrp_instance inside_network {
           # 当前节点在此虚拟路由器中的初始状态（主/备）：MASTER | BACKUP
           state MASTER

           # 内网网卡： vrrp实例工作的网络接口
           interface eth0

           # VRRP 虚拟MAC
           use_vmac <VMAC_INTERFACE>

           # Send/Recv VRRP messages from base interface instead of
           # VMAC interface
           vmac_xmit_base

           # Ignore VRRP interface faults (default unset)
           dont_track_primary

           # optional, monitor these as well.
           # go to FAULT state if any of these go down.
           track_interface {
             eth0
             eth1
             ...
           }

           # default IP for binding vrrpd is the primary IP
           # on interface. If you want to hide location of vrrpd,
           # use this IP as src_addr for multicast or unicast vrrp
           # packets. (since it's multicast, vrrpd will get the reply
           # packet no matter what src_addr is used).
           # optional
           mcast_src_ip <IPADDR>
           unicast_src_ip <IPADDR>

           # Do not send VRRP adverts over VRRP multicast group.
           # Instead it sends adverts to the following list of
           # ip addresses using unicast design fashion. It can
           # be cool to use VRRP FSM and features in a networking
           # environement where multicast is not supported !
           # IP Addresses specified can IPv4 as well as IPv6
           unicast_peer {
             <IPADDR>
             ...
           }

           # Binding interface for lvs syncd
           lvs_sync_daemon_interface eth1

           # delay for gratuitous ARP after transition to MASTER
           garp_master_delay 10 # secs, default 5

           #  虚拟路由器ID，范围0-255; 
           virtual_router_id 51

           # 当前物理节点在此虚拟路由器中的优先级
           priority 100

           # VRRP 通告的时间间隔，秒
           advert_int 1

           # 认证机制
           authentication {
               # PASS与|AH：所有机器保持统一
               # 1：PASS：简单密码：仅前8位有效
               #:2：AH - IPSEC (不推荐)
               auth_type PASS
               auth_pass 1234
           }

           #addresses add|del on change to MASTER, to BACKUP.
           #With the same entries on other machines,
           #the opposite transition will be occuring.
           virtual_ipaddress {
               <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
               192.168.200.17/24 dev eth1
               192.168.200.18/24 dev eth2 label eth2:1
           }

           #VRRP IP excluded from VRRP
           #optional.
           #For cases with large numbers (eg 200) of IPs
           #on the same interface. To decrease the number
           #of packets sent in adverts, you can exclude
           #most IPs from adverts.
           #The IPs are add|del as for virtual_ipaddress.
           virtual_ipaddress_excluded {
            <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE>
            <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE>
               ...
           }
           # routes add|del when changing to MASTER, to BACKUP
           virtual_routes {
               # src <IPADDR> [to] <IPADDR>/<MASK> via|gw <IPADDR> [or <IPADDR>] dev <STRING> scope <SCOPE> tab
               src 192.168.100.1 to 192.168.109.0/24 via 192.168.200.254 dev eth1
               192.168.110.0/24 via 192.168.200.254 dev eth1
               192.168.111.0/24 dev eth2
               192.168.112.0/24  via  192.168.100.254       192.168.113.0/24  via  192.168.200.254  or  192.168.100.254  dev eth1
            blackhole 192.168.114.0/24
           }

           # VRRP will normally preempt a lower priority
           # machine when a higher priority machine comes
           # online.  "nopreempt" allows the lower priority
           # machine to maintain the master role, even when
           # a higher priority machine comes back online.
           # NOTE: For this to work, the initial state of this
           # entry must be BACKUP.
           nopreempt

           # Seconds after startup until preemption
           # (if not disabled by "nopreempt").
           # Range: 0 (default) to 1,000
           # NOTE: For this to work, the initial state of this
           # entry must be BACKUP.
           preempt_delay 300    # waits 5 minutes

           # Debug level, not implemented yet.
           debug

           # notify scripts, alert as above
           notify_master <STRING>|<QUOTED-STRING>
           notify_backup <STRING>|<QUOTED-STRING>
           notify_fault <STRING>|<QUOTED-STRING>
           notify <STRING>|<QUOTED-STRING>
           smtp_alert
        }







LVS CONFIGURATION
       contains subblocks of Virtual server group(s) and Virtual server(s)

       The subblocks contain arguments for ipvsadm(8).  A knowlege of ipvsadm(8) will be helpful here.

Virtual server group(s)
        # optional
        # this groups allows a service on a real_server
        # to belong to multiple virtual services
        # and to be only health checked once.
        # Only for very large LVSs.
        virtual_server_group <STRING> {
               #VIP port
               <IPADDR> <PORT>
               <IPADDR> <PORT>
               ...
               #
               # <IPADDR RANGE> has the form
               # XXX.YYY.ZZZ.WWW-VVV eg 192.168.200.1-10
               # range includes both .1 and .10 address
               <IPADDR RANGE> <PORT># VIP range VPORT
               <IPADDR RANGE> <PORT>
               ...
               fwmark <INT>  # fwmark
               fwmark <INT>
               ...  }


Virtual server(s)
       A virtual_server can be a declaration of one of

       vip vport (IPADDR PORT pair)

       fwmark <INT>

       (virtual server) group <STRING>

                  #setup service
                  virtual_server IP port |
                  virtual_server fwmark int |
                  virtual_server group string
                  {
                  # delay timer for service polling
                  delay_loop <INT>

                  # LVS scheduler
                  lb_algo rr|wrr|lc|wlc|lblc|sh|dh
                  # Enable One-Packet-Scheduling for UDP (-O in ipvsadm)
                  ops
                  # LVS forwarding method
                  lb_kind NAT|DR|TUN
                  # LVS persistence timeout, sec
                  persistence_timeout <INT>
                  # LVS granularity mask (-M in ipvsadm)
                  persistence_granularity <NETMASK>
                  # Only TCP is implemented
                  protocol TCP
                  # If VS IP address is not set,
                  # suspend healthchecker's activity
                  ha_suspend

                  # VirtualHost string for HTTP_GET or SSL_GET
                  # eg virtualhost www.firewall.loc
                  virtualhost <STRING>

                  # Assume silently all RSs down and healthchecks
                  # failed on start. This helps preventing false
                  # positive actions on startup. Alpha mode is
                  # disabled by default.
                  alpha

                  # On daemon shutdown, consider quorum and RS
                  # down notifiers for execution, where appropriate.
                  # Omega mode is disabled by default.
                  omega

                  # Minimum total weight of all live servers in
                  # the pool necessary to operate VS with no
                  # quality regression. Defaults to 1.
                  quorum <INT>

                  # Tolerate this much weight units compared to the
                  # nominal quorum, when considering quorum gain
                  # or loss. A flap dampener. Defaults to 0.
                  hysteresis <INT>

                  # Script to launch when quorum is gained.
                  quorum_up <STRING>|<QUOTED-STRING>

                  # Script to launch when quorum is lost.
                  quorum_down <STRING>|<QUOTED-STRING>


                  # setup realserver(s)

                  # RS to add when all realservers are down
                  sorry_server <IPADDR> <PORT>
                  # applies inhibit_on_failure behaviour to the
                  # preceding sorry_server directive
                  sorry_server_inhibit

                  # one entry for each realserver
                  real_server <IPADDR> <PORT>
                     {
                         # relative weight to use, default: 1
                         weight <INT>
                         # Set weight to 0
                         # when healthchecker detects failure
                         inhibit_on_failure

                         # Script to launch when healthchecker
                         # considers service as up.
                         notify_up <STRING>|<QUOTED-STRING>
                         # Script to launch when healthchecker
                         # considers service as down.
                         notify_down <STRING>|<QUOTED-STRING>

                         # pick one healthchecker
                         # HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK

                         # HTTP and SSL healthcheckers
                         HTTP_GET|SSL_GET
                         {
                             # A url to test
                             # can have multiple entries here
                             url {
                               #eg path / , or path /mrtg2/
                               path <STRING>
                               # healthcheck needs status_code
                               # or status_code and digest
                               # Digest computed with genhash
                               # eg digest 9b3a0c85a887a256d6939da88aabd8cd
                               digest <STRING>
                               # status code returned in the HTTP header
                               # eg status_code 200
                               status_code <INT>
                             }
                             # number of get retry
                             nb_get_retry <INT>
                             # delay before retry
                             delay_before_retry <INT>

                             # ======== generic connection options
                             # Optional IP address to connect to.
                             # The default is real server's IP
                             connect_ip <IP ADDRESS>
                             # Optional port to connect to if not
                             # The default is real server's port
                             connect_port <PORT>
                             # Optional interface to use to
                             # originate the connection
                             bindto <IP ADDRESS>
                             # Optional source port to
                             # originate the connection from
                             bind_port <PORT>
                             # Optional connection timeout in seconds.
                             # The default is 5 seconds
                             connect_timeout <INTEGER>
                             # Optional fwmark to mark all outgoing
                             # checker pakets with
                             fwmark <INTEGER>

                             # Optional random delay to begin initial check for
                             # maximum N seconds.
                             # Useful to scatter multiple simultaneous
                             # checks to the same RS. Enabled by default, with
                             # the maximum at delay_loop. Specify 0 to disable
                             warmup <INT>
                         } #HTTP_GET|SSL_GET

                         #TCP healthchecker (bind to IP port)
                         TCP_CHECK
                         {
                         } #TCP_CHECK

                         # SMTP healthchecker
                         SMTP_CHECK
                         {
                         } #SMTP_CHECK

                         #MISC healthchecker, run a program
                         MISC_CHECK
                         {
                         }
                     } # realserver defn
                  } # virtual service













                            
                            virtual_ipaddress { # 定义虚拟IP
                                <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
                                192.168.200.17/24 dev eth1
                                192.168.200.18/24 dev eth2 label eth2:1
                            }
                            
                            track_interface {# 定义要监控的接口
                                eth0
                                eth1
                                ...
                            }
                    
                            nopreempt：非抢占模式
                            
                            通告脚本定义
                            notify_master <STRING>|<QUOTED-STRING>
                            notify_backup <STRING>|<QUOTED-STRING>
                            notify_fault <STRING>|<QUOTED-STRING>
                            notify <STRING>|<QUOTED-STRING>
                            
                        通知脚本示例：
                            #!/bin/bash
                            #
                            contact='root@localhost'

                            notify() {
                            mailsubject="$(hostname) to be $1: vip floating"
                            mailbody="$(date +'%F %T'): vrrp transition, $(hostname) change to be $1"
                            echo $mailbody | mail -s "$mailsubject" $contact
                            }

                            case $1 in
                            master)
                                notify master
                                ;;
                            backup)
                                notify backup
                                ;;
                            fault)
                                notify fault
                                ;;
                            *)
                                echo "Usage: $(basename $0) {master|backup|fault}"
                                ;;
                            esac                
                        
                        调用方法：
                            vrrp_instance <STRING> {
                                ...
                                notify_master "/etc/keepalived/notify.sh master"
                                notify_backup "/etc/keepalived/notify.sh backup"
                                notify_fault "/etc/keepalived/notify.sh fault"              
                            }
                            
                            注意：要使用双引号；
                
                虚拟服务器：
                    virtual_server IP PORT |
                    virtual_server FWM # {
                        lb_algo rr|wrr|lc|wlc|lblc|sh|dh
                            定义负载均衡调度方法；
                        delay_loop <INT>
                            定义服务轮询时间间隔；健康检测时间。
                        lb_kind NAT|DR|TUN
                            集群的类型；
                        persistence_timeout <INT>
                            持久连接时长； 
                        protocol TCP
                            服务协议；
                        sorry_server <IPADDR> <PORT>
                            所有RS均故障时，提供say sorry的服务器；
                        real_server <IPADDR> <PORT> {
                            weight <INT>
                                权重
                            notify_up <STRING>|<QUOTED-STRING>
                                节点上线时调用通知脚本；
                            notify_down <STRING>|<QUOTED-STRING>
                                节点离线时调用的通知脚本；
                            # HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK
                            # 支持的所有健康状态检测方式；
                            
                            HTTP_GET|SSL_GET {
                                 url {
                                    path <STRING>
                                        健康状态检测时请求的资源的URL；
                                    digest <STRING>      
                                        基于获取的内容摘要码进行健康状态判定；
                                    status_code <INT>
                                }
                                nb_get_retry <INT>
                                    尝试的次数；
                                delay_before_retry <INT>
                                    两次尝试之间的时间间隔
                                connect_timeout <INTEGER>
                                    连接的超时时长； 
                                connect_ip <IP ADDRESS>
                                    向此处指定的地址发测试请求；
                                connect_port <PORT>
                                    向此处指定的PORT发测试请求；
                                bindto <IP ADDRESS>
                                    指定测试请求报文的源IP；
                                bind_port <PORT>
                                    指定测试请求报文的源PORT；
                            }
                            
                            TCP_CHECK {
                                ...
                            }
                            
            实践作业：
                keepalived高可用ipvs集群； 
                ipvs集群提供php应用，例如phpwind；
        
回顾：
    HA Cluster:
        vrrp：keepalived
        ais: heartbeat, corosync
        
    keepalived:
        vrrp：Virtual Redundant Routing Protocol
            主/备，主/主（主/备，备/主）
        ipvs wrapper， 
        checker
            HTTP_GET, SSL_GET, TCP_CHECK
            
keepalived(2)：

    vrrp_script：自定义一个资源监控脚本； vrrp实例能根据脚本状态返回值来；
        公共定义，可被多个实例调用，因此，定义在vrrp实例之外；
    
    track_script：调用vrrp_script定义的脚本去监控资源；
        定义在实例之内，调用事先定义好的vrrp_script；
        
    使用示例1：
        vrrp_script chk_down {
            script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
            interval 2
            weight -5
        }

        track_script {
            chk_down
        }
        
    使用示例2：监控具体的服务进程
        vrrp_script chk_httpd {
            script "killall -0 httpd"
            interval 2
            weight -5
        }       
        
        track_script {
            chk_httpd
        }           
    
        通知脚本示例：
            #!/bin/bash
            #
            contact='root@localhost'

            notify() {
            mailsubject="$(hostname) to be $1: vip floating"
            mailbody="$(date +'%F %T'): vrrp transition, $(hostname) change to be $1"
            echo $mailbody | mail -s "$mailsubject" $contact
            }

            case $1 in
            master)
                notify master
                systemctl  start  httpd.service
                ;;
            backup)
                notify backup
                systemctl  restart httpd.service
                ;;
            fault)
                notify fault
                ;;
            *)
                echo "Usage: $(basename $0) {master|backup|fault}"
                ;;
            esac            
        
    双主模型：
        主/备，备/主
        
        node1：
            vrrp_instance VI_1 {
            state MASTER
            interface eno16777736
            virtual_router_id 171
            priority 100
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass 0c5UMiJo
            }

            virtual_ipaddress {
                172.18.100.77 dev eno16777736 label eno16777736:0
            }

            track_script {
                chk_down
            }

            notify_master "/etc/keepalived/notify.sh master"
            notify_backup "/etc/keepalived/notify.sh backup"
            notify_fault "/etc/keepalived/notify.sh fault"
            }

            vrrp_instance VI_2 {
            state BACKUP
            interface eno16777736
            virtual_router_id 172
            priority 98
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass 0c6UMiJo
            }

            virtual_ipaddress {
                172.18.100.78 dev eno16777736 label eno16777736:1
            }

            track_script {
                chk_down
            }

            notify_master "/etc/keepalived/notify.sh master"
            notify_backup "/etc/keepalived/notify.sh backup"
            notify_fault "/etc/keepalived/notify.sh fault"
            }
            
        node2:
            vrrp_instance VI_1 {
            state BACKUP
            interface eno16777736
            virtual_router_id 171
            priority 98
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass 0c5UMiJo
            }

            virtual_ipaddress {
                172.18.100.77 dev eno16777736 label eno16777736:0
            }

            track_script {
                chk_down
            }

            notify_master "/etc/keepalived/notify.sh master"
            notify_backup "/etc/keepalived/notify.sh backup"
            notify_fault "/etc/keepalived/notify.sh fault"
            }

            vrrp_instance VI_2 {
            state MASTER
            interface eno16777736
            virtual_router_id 172
            priority 100
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass 0c6UMiJo
            }

            virtual_ipaddress {
                172.18.100.78 dev eno16777736 label eno16777736:1
            }

            track_script {
                chk_down
            }

            notify_master "/etc/keepalived/notify.sh master"
            notify_backup "/etc/keepalived/notify.sh backup"
            notify_fault "/etc/keepalived/notify.sh fault"
            }
        
        
        
    博客作业：keepalived单实例，高可用lvs，vrrp_script高可用nginx，双主nginx；
    

    
                
        
            
            
~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.200.100:443 rr persistent 50
  -> 192.168.201.100:443          Masq    1      0          0         
TCP  10.10.10.2:1358 rr persistent 50
  -> 192.168.200.2:1358           Masq    1      0          0         
  -> 192.168.200.3:1358           Masq    1      0          0         
TCP  10.10.10.3:1358 rr persistent 50
  -> 192.168.200.4:1358           Masq    1      0          0         
  -> 192.168.200.5:1358           Masq    1      0          0         
















http://www.cnblogs.com/mchina/archive/2012/08/27/2644391.html
http://bbs.linuxtone.org/forum.php?mod=viewthread&tid=17719
http://blog.chinaunix.net/uid-27571599-id-3733849.html